Affordance of vibrational excitation for music composition and performance

Mechanical vibrations have typically been used in the performance domain within feedback systems to inform musicians of system states or as communication channels between performers. In this paper, we propose the addi- tional taxonomic category of vibrational excitation of mu- sical instruments for sound generation. To explore the va- riety of possibilities associated with this extended taxon- omy, we present the Oktopus, a multi-purpose wireless sys- tem capable of motorised vibrational excitation. The sys- tem can receive up to eight inputs and generates vibrations as outputs through eight motors that can be positioned ac- cordingly to produce a wide range of sounds from an ex- cited instrument. We demonstrate the usefulness of the proposed system and extended taxonomy through the de- velopment and performance of Live Mechanics, a compo- sition for piano and interactive electronics

Improving peak picking using multiple time-step loss functions

The majority of state-of-the-art methods for music infor-mation retrieval (MIR) tasks now utilise deep learningmethods reliant on minimisation of loss functions such ascross entropy. For tasks that include framewise binaryclassification (e.g., onset detection, music transcription)classes are derived from output activation functions byidentifying points of local maxima, or peaks. However, theoperating principles behind peak picking are different tothat of the cross entropy loss function, which minimises theabsolute difference between the output and target valuesfor a single frame. To generate activation functions moresuited to peak-picking, we propose two versions of a newloss function that incorporates information from multipletime-steps: 1)multi-individual, which uses multiple indi-vidual time-step cross entropies; and 2)multi-difference,which directly compares the difference between sequentialtime-step outputs. We evaluate the newly proposed lossfunctions alongside standard cross entropy in the popularMIR tasks of onset detection and automatic drum tran-scription. The results highlight the effectiveness of theseloss functions in the improvement of overall system ac-curacies for both MIR tasks. Additionally, directly com-paring the output from sequential time-steps in the multi-difference approach achieves the highest performance

Onset Detection for String Instruments Using Bidirectional Temporal and Convolutional Recurrent Networks

Recent work in note onset detection has centered on deep learning models such as recurrent neural networks (RNN), convolutional neural networks (CNN) and more recently temporal convolutional networks (TCN), which achieve high evaluation accuracies for onsets characterized by clear, well-defined transients, as found in percussive instruments. However, onsets with less transient presence, as found in string instrument recordings, still pose a relatively difficult challenge for state-of-the-art algorithms. This challenge is further exacerbated by a paucity of string instrument data containing expert annotations. In this paper, we propose two new models for onset detection using bidirectional temporal and recurrent convolutional networks, which generalise to polyphonic signals and string instruments. We perform evaluations of the proposed methods alongside state-of-the-art algorithms for onset detection on a benchmark dataset from the MIR community, as well as on a test set from a newly proposed dataset of string instrument recordings with note onset annotations, comprising approximately 40 minutes and over 8,000 annotated onsets with varied expressive playing styles. The results demonstrate the effectiveness of both presented models, as they outperform the state-of-the-art algorithms on string recordings while maintaining comparative performance on other types of music

Games without frontiers: Audio games for music production and performance

The production of electronic dance music most often takes the form of sound-events arranged on a timeline, rendered as a stereo recording and mixed with other recordings in the context of mixes. Live electronic dance music on the other hand involves interaction with the sound-events in real time, but nonetheless subject to a master timekeeper. A similar comparison exists between sound for film and sound for (video) games, however in the case of games there is no assumption of a master timekeeper, and the timing of events is relative to the actions of the player and the state of the game environment. The authors explain a method by which electronic dance music can be produced in a similar manner to producing game audio, and in fact that an interactive or live piece can be considered an audio game. Musical structure is composed conceptually as scenes in a film, where interacting sound-machines generate rhythmic patterns of sound-events that place the performer/player in a virtual space. The performer/player pursues musical goals in nonlinear time while maintaining the ability to sequentially arrange pieces in a coherent mix

A plugin for neural audio synthesis of impact sound effects

The term impact sound as referred to in this paper, can be broadly defined as the sudden burst of short-lasting impulsive noise generated by the collision of two objects. This type of sound effect is prevalent in multimedia productions. However, conventional methods of sourcing these materials are often costly in time and resources. This paper explores the potential of neural audio synthesis for generating realistic impact sound effects, targeted for use in multimedia such as films, games, and AR/VR. The designed system uses a Realtime Audio Variational autoEncoder (RAVE) [2] model trained on a dataset of over 3,000 impact sound samples for inference in a Digital Audio Workstation (DAW), with latent representations exposed as user controls. The performance of the trained model is assessed using various objective evaluation metrics, revealing both the prospects and limitations of this approach. The results and contributions of this paper are discussed, with audio examples and source code made available

Trainable data manipulation with unobserved instruments

Machine learning algorithms are the core components in a wide range of intelligent music production systems. As training data for these tasks is relatively sparse, data augmentation is often used to generate additional training data by slightly altering existing training data. User-defined techniques require a long parameter tuning process and typically use a single set of global variables. To address this, a trainable data manipulation system, termed player vs transcriber, was proposed for the task of automatic drum transcription. This paper expands the player vs transcriber model by allowing unobserved instruments to also be manipulated within the data augmentation and sample addition stages. Results from two evaluations demonstrate that this improves performance and suggests that trainable data manipulation could benefit additional intelligent music production tasks